feat(bench): add TPC-DS benchmarks #3845
Conversation
|
ACTION NEEDED delta-rs follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
abhiaagarwal
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3845 +/- ##
==========================================
- Coverage 73.78% 73.77% -0.02%
==========================================
Files 151 151
Lines 39190 39168 -22
Branches 39190 39168 -22
==========================================
- Hits 28917 28895 -22
- Misses 8981 8983 +2
+ Partials 1292 1290 -2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Awesome @abhiaagarwal!
Read through the code and looks great. Really stepping up the benchmarking game here!
There are a few minor CI failures to fix, but we can merge otherwise.
One question that came up, especially considering @alexwilcoxson-rel latest PR. Is there a way we can track additional metrics we collect during query execution as benchmarking results? I.e. teacking how well we do file skipping / reading so files skipped or bytes read ...
Signed-off-by: Abhi Agarwal <[email protected]>
Signed-off-by: Abhi Agarwal <[email protected]>
Signed-off-by: Abhi Agarwal <[email protected]>
Signed-off-by: Abhi Agarwal <[email protected]>
Signed-off-by: Abhi Agarwal <[email protected]>
abhiaagarwal
force-pushed
the
abhi/new-benchmarks
branch
from
75c8461 to
82e2b8b
Compare
I looked into this, and I believe not :( I believe this issue in divan would be what we're looking for nvzqz/divan#34 |
3 participants
Description
Adds benchmarks based on the TPS-DS queries used in the delta-spark repository. This is definitely closer to a "datafusion benchmark" over a "delta benchmark", but it provides a decent baseline for optimization opportunities.
Related Issue(s)
Progress towards #3843